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Abstract 


System-of-systems  (SoS)  acquisition  research  has  identified  lack  of  alignment 
and  lack  of  collaboration  as  two  important  issues  leading  to  problems  in  SoS 
acquisition.  This  report  captures  the  exploratory  work  toward  improving  alignment 
between  and  collaboration  among  the  individual  system  programs  in  the 
development  of  an  SoS.  An  SoS  inter-program  collaboration  approach  is  proposed. 

It  is  inspired  by  some  existing  web-based  collaborative  systems,  such  as  eBay, 
Facebook,  and  Eureka,  and  suggests  an  attraction  mechanism  to  effect  SoS  inter¬ 
program  collaboration.  In  addition,  a  web-based  collaborative  system  is  also 
suggested.  Based  on  an  architecture  for  distributed  and  interoperable  management 
of  multi-site  production  projects,  it  allows  personnel  of  all  programs  associated  with 
an  SoS  to  input  need  points  for  component  system  inputs  and  retrieve  information 
required  to  align  the  individual  programs.  Furthermore,  contracting  structures  for 
facilitating  collaboration  among  the  system  programs  are  also  considered.  Finally, 
this  work  forms  a  basis  for  implementing  a  web-based  SoS  collaborative  system  to 
support  Department  of  Defense  (DoD)  SoS  acquisition  programs. 

Keywords:  System  of  systems  (SoS),  inter-program  collaboration,  inter- 
organizational  collaboration,  web-based  collaborative  system,  contracting  structures, 
SoS  acquisition,  attractor  mechanism,  emergent  behavior,  collaborative  capacity, 
SCEP  model 
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Introduction 


The  most  common  type  of  Department  of  Defense  (DoD)  systems  of  systems 
(SoS)  development  is  one  in  which  an  SoS  is  created  by  integrating  separately 
developed  systems — legacy  systems,  developmental  systems,  or  some  combination 
of  both.  Research  in  SoS  acquisition  has  identified  lack  of  alignment  and  lack  of 
collaboration  as  two  important  issues  leading  to  problems  in  SoS  acquisition.  Lack 
of  alignment  means  a  system  is  not  ready  for  its  integration  into  an  SoS  or,  because 
of  the  lack  of  the  front-end  SoS  systems  engineering  (SE),  the  SoS  integration 
discovers  that  the  system  does  not  meet  the  performance  requirements  or  the 
interface  requirements.  Lack  of  collaboration  means  the  individual  system  programs 
fail  to  work  with  each  other  to  achieve  the  goals  of  the  SoS  program. 

SoS  acquisition  requires  the  availability  of  surrogates  of  component  systems, 
and  later  of  the  “as-built”  component  systems,  in  a  timely  manner  in  order  to  support 
SoS  integration  testing.  However,  the  acquisition  schedules  for  the  component 
systems  are  typically  developed  independently  of  the  SoS  development  schedule. 
There  is,  thus,  no  assurance  that  the  SoS  integration  testing  can  be  completed  as 
planned,  resulting  in  a  slip  of  the  SoS  acquisition  schedule  and  an  associated  cost 
overrun.  Even  when  the  schedules  are  aligned,  a  lack  of  the  front-end  SoS  SE  may 
cause  a  system  to  not  meet  the  performance  or  interface  requirements  during  the 
SoS  integration  or  there  may  be  misalignment  of  resources  to  support  SoS 
integration  testing,  such  as,  for  example,  the  absence  of  component  system  experts 
to  support  SoS  integration  testing. 

The  lack  of  alignment  is  related  not  only  to  the  front-end  SoS  SE  in  the  SoS 
acquisition,  but  also  to  the  lack  of  collaboration.  Collaboration  in  the  development  of 
an  SoS  is  multi-dimensional;  that  is,  it  must  exist  between  DoD  system  program 
offices,  between  contractors,  and  between  DoD  program  offices  and  contractors. 
“Inter-organizational  collaboration  has  been  cited  as  a  critical  requirement  for 
successful  outcomes;  and  for  those  agencies  struggling  to  achieve  their  goals,  lack 
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of  inter-organizational  collaboration  has  been  cited  as  a  factor  accounting  for  failure” 
(Kirschman  &  LaPorte,  2008).  Inter-organizational  collaboration  requires 
collaborative  capacity.  Mirroring  the  definition  of  collaborative  capacity  by  Hocevar, 
Jansen,  and  Thomas  (2007),  collaborative  capacity  in  SoS  acquisition  is  defined  as 
the  ability  of  the  individual  system  programs  to  enter  into,  develop,  and  sustain  inter¬ 
system  programs  in  the  pursuit  of  SoS  collective  outcomes.  Such  collaborative 
capacity  is  needed,  in  addition  to  contracting  structure  and  organizational  structures 
(Rendon,  Huynh,  &  Osmundson,  2010;  Huynh,  Rendon,  &  Osmundson,  2010),  to 
effect  resolution  of  the  SoS  acquisition  issues  raised  in  Osmundson,  Huynh  and 
Langford  (2007).  These  issues  are  initial  agreement,  SoS  control,  organizing, 
staffing,  team  building,  training  data  requirements,  interfaces,  risk  management  at 
the  SoS  level,  SoS  testing,  measures  of  effectiveness,  and  emergent  behavior.  For 
this  report  to  be  self-contained,  a  brief  explanation  of  these  issues,  excerpted  from 
Rendon,  Huynh,  &  Osmundson  (2010)  and  Huynh,  Rendon,  &  Osmundson  (2010), 
follows. 


Initial  agreement  refers  to  decision  makers  initially  getting  agreement 
that  an  SoS  meets  some  desirable  objectives.  It  is  an  issue  in 
particular  when  the  SoS  involves  systems  from  different  organizations 
or  services  because  establishing  an  initial  agreement  is  contingent  on 
quantifying  the  benefits  and  risks  of  the  new  SoS. 

SoS  control  must  be  established:  Who  will  control  the  SoS  and  how  it 
will  be  controlled.  Each  partner  may  lose  some  measure  of  control  over 
its  own  systems  in  order  to  enable  overall  SoS  control. 

Organizing  is  a  key  issue  of  how  to  organize  for  the  development  and 
operation  of  an  SoS.  An  example  is  the  systems  engineering  process: 
How  are  processes  that  interface  with  SoS  processes  established  and 
monitored? 

Staffing,  team  building,  and  training  refer  to  how  an  SoS  will  be  staffed 
and  operated.  SoS  operations  must  be  planned  for,  the  skills  required 
for  SoS  operations  identified,  and  personnel  with  the  proper  skills 
acquired  and  trained  in  SoS  operations. 

Data  requirements  is  an  issue  concerning  sharing  of  classified  and/or 
proprietary  design  information  among  the  SoS  partners,  who  must 
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recognize  and  weigh  a  possible  loss  of  their  system’s  operational 
superiority  based  on  the  shared  classified  or  proprietary  design 
information  against  the  SoS  benefits. 

■  Interfaces  must  be  identified  and  managed.  Common  language, 
grammar,  and  usage  must  be  established  (for  information  SoSs); 
configuration  management  invoked  to  assure  common  agreements  are 
followed;  and  required  information  security  levels  identified  and 
provisions  made  to  assure  meeting  of  security  requirements. 

■  Risk  management  at  the  SoS  level  is  an  issue  related  to  the  mitigation 
of  SoS  risks  potentially  affected  by  component  systems;  the  risk 
mitigation  requires  detailed  knowledge  of  component  system  risks  and 
variations  in  individual  system  outputs. 

■  SoS  testing  requires  that  each  SoS  partner’s  system  be  tested  in  a 
manner  that  resolves  any  of  its  concerns  about  operational  behavior 
and  that  SoS  threads  be  tested. 

■  Measures  of  effectiveness  is  an  issue  because  their  strong 
dependence  on  individual  component  systems’  measures  of 
performance  requires  an  understanding  of  the  latter,  and  this  issue  is 
related  to  the  issues  of  data  requirements  and  interfaces. 

■  Emergent  behavior,  exhibited  by  the  SoS,  resulting  from  unknown 
interactions  among  the  constituent  systems  or  from  its  interaction  with 
the  environment,  need  to  be  collectively  understood,  analyzed,  and 
resolved,  in  particular  when  an  emergent  behavior  may  be  detrimental 
to  one  or  more  of  the  partners. 

The  SoS  acquisition  issues  addressed  in  this  report  are  not  just  the  ability  of 
individual  system  programs  to  enter  into,  develop,  and  sustain  inter-systems 
programs,  but  also  the  approach  to  and  mechanism  of  inducing  or  motivating  the 
individual  system  programs  to  develop  and  maintain  such  an  ability.  The 
mechanism  is  intended  to  remove  barriers  against  and  implement  factors  favorable 
to  the  realization  of  collaborations  among  the  individual  system  programs.  The 
approach  proposed  in  this  work  to  bring  about  collaboration  among  the  individual 
system  programs  is  to  combine  this  mechanism  with  a  web-based  system  to  aid  in 
managing  their  collaboration  as  well  as  to  implement  a  front-end  SoS  SE  in  the  SoS 
acquisition.  As  the  lack  of  alignment  is  tied  to  both  the  lack  of  the  front-end  SoS  SE 
in  the  SoS  acquisition  and  the  lack  of  collaboration,  the  collaboration  brought  about 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-3- 


by  this  approach  in  turn  aids  in  improving  the  alignment  of  the  individual  system 
programs. 


Due  to  constraints  in  the  scope  of  this  report,  the  front-end  SoS  SE  in  the  SoS 
acquisition  is  not  discussed  here.  This  report  is  focused  only  on  collaboration 
among  the  individual  system  programs  as  it  is  related  to  the  misalignment  issue.  A 
discussion  of  front-end  SoS  SE  in  the  SoS  acquisition  can  be  found  in  Huynh, 
Rendon,  and  Osmundson  (2010).  .  Furthermore,  Heng  (2011)  conducted  a 
quantitative  analysis  of  the  benefits  of  having  the  front-end  SoS  SE  in  the  SoS 
acquisition. 

Enhancement  of  program  collaborations  might  include  re-organization  of 
program  structures,  creation  of  new  program  structures,  creation  of  new  contracting 
structures,  and  use  of  incentives.  These  techniques,  however,  are  not  necessarily 
the  only  means  to  effect  enhancement  of  program  collaborations.  In  this  work,  the 
key  idea  underlying  the  proposed  approach  is  the  collaborative  behavior  observed 
on  some  existing  web-based  systems.  That  is,  what  has  been  done  with  web-based 
collaborative  systems  is  extended  to  a  web-based  system  that  will  facilitate  the 
development  of  an  SoS  through  collaborative  behavior  from  the  individual  system 
programs.  It  is  the  web-based  system  concept  that  inspires  the  mechanism 
proposed  in  this  research  for  inter-program  collaboration. 

System-of-systems  (SoS)  modeling  and  simulation  has  recently  been  applied 
to  the  problem  of  engineering  SoSs  in  order  to  prevent  undesired  emergent  behavior 
(Osmundson,  2009).  Example  SoSs  that  have  been  studied  are  the  collateralized 
debt  obligation  market  (Osmundson,  Langford,  &  Huynh,  2009)  and  the  North 
American  electric  power  grid  (Osmundson,  Huynh,  &  Langford,  2008).  Theoretical 
studies  of  these  SoSs  have  also  been  carried  out  to  validate  the  results  from  the 
modeling  and  simulation  work  (Huynh  &  Osmundson,  2008;  Huynh  &  Osmundson, 
2009).  The  results  of  these  studies  indicate  that  SoS  modeling  and  simulation  can 
be  used,  at  least  in  some  cases,  to  predict  undesired  emergent  behavior  in  SoSs 
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that  consist  of  engineered  systems  and  non-engineered  systems,  including  people, 
and  to  identify  ways  to  prevent  or  mitigate  undesired  behavior. 


Essentially,  to  deal  with  the  lack  of  alignment  and  collaboration  in  SoS 
acquisition,  an  SoS  acquisition  program  needs  to  institute  an  overarching  front-end 
SoS  SE  in  the  SoS  acquisition  program  and  to  implement  an  approach  to  achieving 
collaboration  among  the  individual  system  programs. 

In  this  report,  a  web-based  collaborative  system  (WBCS)  is  proposed,  on 
which  personnel  of  all  programs  associated  with  an  SoS  can  input  and  retrieve 
information  required  to  align  the  individual  programs.  As  discussed  in  detail  later  in 
this  report,  the  proposed  collaborative  web-based  system  is  based  on  the  concept  of 
an  architecture  for  distributed  and  interoperable  management  of  multi-site  production 
projects  espoused  in  Ishak,  Archimede,  and  Charbonnaud  (2010).  The  kernel  of 
this  system  is  the  SCEP  (Supervisor,  Customer,  Environment,  Producer)  model 
(Archimede  &  Coudert,  2001).  Based  on  multi-agent  systems  (Ferber,  1999),  the 
SCEP  allows  a  distributed  management  of  acquisition  programs  (i.e.,  system 
programs  and  the  SoS  program)  and  their  cooperation  via  a  shared  environment. 

The  overall  development  of  the  SoS  and  component  systems  is  treated  as  a  network 
and  the  need  points  for  component  system  inputs  are  identified  as  intermediate 
milestones  requiring  SoS-component  system  collaboration.  An  attraction 
mechanism  to  effect  SoS  inter-program  collaboration  is  incorporated  in  this  web- 
based  SoS  collaborative  system. 

The  purposes  of  this  report  are  to 


discuss  in  some  detail  some  existing  web-based  collaborative  systems; 

explain  our  exploratory  work  toward  improving  alignment  between  and 
collaboration  among  the  individual  system  programs  in  the 
development  of  a  system  of  systems; 

elucidate  the  approach  proposed  in  this  research  for  achieving 
collaboration  among  the  individual  system  programs;  and 
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■  discuss  contracting  structures  motivating  or  facilitating  collaboration 
among  the  system  programs. 

The  rest  of  the  report  is  organized  as  follows.  First,  the  web-based  collaborative 
systems  are  described  and  explained.  Modeling  and  simulation  of  the  web-based 
collaborative  systems  are  discussed  next.  The  SoS  inter-program  collaboration 
approach  is  then  explained.  A  discussion  of  contracting  structures  for  facilitating 
collaboration  among  the  system  programs  follows.  Finally,  the  report  ends  with 
some  concluding  remarks. 
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Web-Based  Collaborative  Systems 


A.  The  Underlying  Idea  of  Web-Based  Collaborative  Systems 

Many  web-based  systems  are  based  on  what  is  known  as  network  effect 
(Liebowitz  &  Margolis,  1995).  When  the  network  effect  is  present,  the  value  of  the 
system  to  customers  or  collaborators  is,  thus,  dependent  on  the  number  of 
customers  or  collaborators  already  using  the  system. 

Network  effects  become  significant  after  a  certain  number  of  people  have 
subscribed  to  the  system,  called  the  critical  mass.  At  the  critical  mass  point,  the 
value  obtained  from  the  good  or  service  is  greater  than  or  equal  to  the  price  paid  for 
the  good  or  service.  Cost  is  also  incurred  in  using  a  web-based  system.  Cost  could 
be  the  payment  of  money  for  a  service  or  product,  time  to  prepare  inputs  for  the 
system,  time  spent  using  the  system  before  a  match  is  found,  or  a  loss  associated 
with  the  risk  of  using  the  system,  such  as  not  receiving  goods  paid  for,  receiving 
incorrect  goods,  or  some  other  loss.  There  may  also  be  some  cost  associated  with 
attracting  the  participants.  At  the  critical  mass  point,  the  value  obtained  from  the 
system  is  greater  than  or  equal  to  the  cost  encountered  when  obtaining  the  good  or 
service  provided  by  the  system.  As  the  value  of  the  good  is  determined  by  the  user 
base,  this  implies  that  after  a  certain  number  of  people  have  subscribed  to  the 
service  or  purchased  the  good,  additional  people,  because  of  the  positive  value/cost 
ratio,  will  subscribe  to  the  service  or  purchase  the  good. 

Prior  to  reaching  the  critical  mass,  and  depending  on  the  system  type,  the 
system  must  attract  early  adopters  by  investment  capital,  incentives,  or  other  means. 
In  the  interim,  before  the  critical  mass  is  achieved,  some  early  adopters  may  drop 
out  of  the  system  because  of  lack  of  perceived  value,  while  others  join  the  system. 
Thus,  the  success  of  a  web-based  system  depends  on  achieving  a  critical  mass  of 
subscribers  before  the  effectiveness  of  attracting  additional  subscribers  to  the 
system  is  exhausted. 
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The  system  factors  that  determine  the  success  or  failure  of  a  web-based 
system  include  the  number  of  subscribers  or  participants  as  a  function  of  time;  the 
factors  that  attract  a  subscriber;  the  factors  that  cause  a  subscriber  to  leave  the 
system;  the  value  of  the  system’s  services  to  the  subscriber/participant;  and  the  cost 
of  the  system’s  services  or  products  to  the  subscriber/participant.  The  term 
“participant”  will  be  used  exclusively  hereafter,  as  the  individual  system  programs 
are  “participants,”  although  in  a  strict  sense  the  term  “subscriber”  more  properly 
refers  to  someone  who  pays  for  a  service,  while  a  participant  refers  to  a  person  who 
invests  time  and  effort  to  obtain  a  product  or  service,  but  does  not  pay  money  for  it. 

B.  Examples  of  Collaborative  Systems 

The  type  of  web-based  system  of  most  interest  is  a  collaborative  enterprise 
whose  success  depends  on  the  number  and  quality  of  the  participants,  but  not  on 
how  much  revenue  the  system  attracts.  Examples  of  this  type  of  system  are  those 
that  are  established  to  facilitate  a  process  through  collaborative  behavior,  such  as 
eBay,  Facebook,  and  the  Xerox  Eureka  system. 

eBay  is  an  online  auction  and  shopping  website  on  which  individuals  and 
businesses  buy  and  sell  a  wide  variety  of  products  and  services.  eBay  was  founded 
in  1995  and  experienced  very  rapid  growth.  By  the  second  year  of  operations,  eBay 
hosted  250,000  online  auctions  and  two  million  online  auctions  the  following  year 
(www.en.wikipedia.or  ).  Facebook  is  a  social  networking  website  that  began  in 
February  2004  and  had  more  than  500  million  participants  by  July  2010  (Facebook 
2011).  Participants  maintain  personal  profiles,  add  people  as  friends,  send 
messages  to  friends,  notify  friends  about  updates  to  their  profile,  and  access  friends’ 
profiles.  The  Eureka  system,  developed  by  Xerox  (Choo,  n.d.),  allows  customer 
service  engineers  for  Xerox’s  family  of  copier  machines  to  share  validated  tips  on 
problems  encountered  and  solutions  to  these  problems.  The  system  is  an  example 
of  a  net-based  community  of  practice  within  an  organization.  Customer  service 
engineers  browse  the  Eureka  system  to  see  if  there  is  a  known  solution  to  a  problem 
that  they  are  encountering.  Five  years  after  its  introduction,  the  Eureka  system  had 
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been  widely  adopted  by  Xerox  technicians  and  has  resulted  in  significant  savings  in 
time  and  parts  costs  (Bobrow  &  Whalen,  2002). 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-9- 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


- 10- 


Modeling  and  Simulation  of  Collaborative 
Systems 


The  SoS  modeling  and  simulation  (M&S)  approach  discussed  in  the 
Introduction  section  is  used  to  model  a  system  of  individual  system  programs 
collaborating  to  form  an  SoS.  This  M&S  approach  has  been  illustrated  with  eBay, 
Facebook,  and  Eureka  (Osmundson  &  Holgerson,  2011).  To  be  self-contained,  this 
report  briefly  discusses  the  M&S  approach  and  results  of  these  collaborative 
systems.  This  M&S  approach  considers  a  collaborative  system  to  consist  of  people, 
databases,  and  other  elements.  People  interact  with  one  another  directly,  through 
databases  and/or  other  elements,  to  achieve  outcomes. 

Three  types  of  discrete-event  models  represent  eBay,  Facebook,  and  the 
Xerox  Eureka  system.  Each  model  assumes  a  specific  type  of  attraction 
mechanism,  unique  to  each  system,  which  attracts  sufficient  users  over  time, 
resulting  in  a  successful  system  whose  value  exceeds  its  costs.  In  Figure  1,  the 
users  interact  with  other  users  and/or  the  web-based  system.  In  each  case,  there  is 
a  small  initial  seed  population  of  users.  If  the  users  are  attracted  to  one  another 
and/or  to  the  system  in  sufficient  numbers,  over  time  a  successful  net  presence 
ensues.  The  key  to  this  type  of  system  is  the  attractor  mechanism,  which  is  the 
mechanism  that  provides  value  to  the  users,  while  at  the  same  time  a  cost  is 
imposed  on  the  users.  The  cost  could  be  a  monetary  fee  and/or — more  likely  in 
many  cases — the  time  and  effort  required  to  participate  in  the  system  and  the 
potential  risk  in  participating  in  the  system.  Each  of  the  models  of  the  three  types  of 
systems  is  implemented  in  Extend,1  a  discrete-event  modeling  and  simulation  tool, 
and  the  results  of  each  of  the  three  types  of  models  agree  closely  with  real-world 
data. 


1  Extend  is  a  product  of  Imagine  That  Inc.,  6830  Via  Del  Oro,  Suite  230,  San  Jose,  CA  95119  USA. 
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Figure  1.  Abstract  Form  of  Web-Based  Systems  Model 

Cost  and  value  are  specific  to  each  example  system.  As  the  eBay  model 
represents  online  sellers  and  buyers  of  a  variety  of  goods,  the  value  to  the  seller  is 
low  cost  of  sales  and,  potentially,  a  large  number  of  buyers,  and  the  value  to  the 
buyer  is  a  wide  selection  of  goods  at  low  prices.  These  values  are  functions  of  the 
number  of  users  over  time;  as  the  number  of  sellers  and  buyers  increases,  the  value 
to  both  parties  increases.  There  are  also  costs  to  the  seller  and  buyer.  The  seller  is 
at  risk  of  not  being  paid  and  the  buyer  is  at  risk  of  not  getting  the  goods  at  all  or  of 
getting  miss-represented  goods  and/or  suffering  identify  theft.  Initially,  these  risks 
were  relatively  high,  but  as  improvements  were  made  to  eBay  over  time,  such  as  the 
introduction  of  seller  ratings  and  use  of  PayPal,  these  risks  declined.  Thus,  the 
value-to-cost  ratio  can  be  represented  by  the  time-dependent  number  of  eBay  users 
and  an  S-curve  function  representing  declining  risk  over  time.  The  rate  at  which 
sellers  enter  the  system  is  dependent  on  the  number  of  buyers  in  the  system.  The 
buyers’  risk  factor  is  given  by  an  S-curve  function.  A  detailed  discussion  of  the 
simulation  results  of  the  Extend  eBay  model,  as  well  as  the  Extend  Facebook  model 
and  the  Extend  Eureka  model,  is  provided  in  Osmundson  and  Holgerson  (2011).  In 
this  report,  it  suffices  to  point  out  the  similarity,  as  shown  in  Figure  2,  between  the 
simulation  results  and  the  eBay  user  growth  data. 
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Figure  2.  Results  of  the  Extend  eBay  Model 

(Site  Analytics,  2010) 

Note.  Red  markers  are  model  results,  and  blue  markers  are  eBay  actual  growth  numbers. 

The  Facebook  model  represents  people  who  want  to  form  social  networks  with  their 
friends.  The  value  to  each  individual  is  the  ability  to  communicate  on  a  regular  basis 
with  a  large  number  of  friends  by  posting  text  and  pictures  to  their  Facebook 
homepage,  which  can  be  viewed  by  their  friends.  Value  increases  with  the  number 
of  friends  added  up  to  a  point  where  the  cost  of  maintaining  meaningful  connections 
is  outweighed  by  the  incremental  value  of  adding  additional  friends  or  becoming  a 
friend  on  another  person’s  site.  There  is  an  initial  population  of  participants,  and 
new  participants  arrive  at  a  rate  proportional  to  the  total  population.  Participants 
look  for  a  match — that  is,  a  friend — and  the  probability  of  finding  a  friend  is 
proportional  to  the  total  population.  As  shown  in  Figure  3,  the  Extend  model  results 
fit  the  actual  Facebook  population  data  fairly  well  through  the  first  41  months,  but 
beyond  that  point  the  model  population  grows  at  a  rate  faster  than  the  actual 
population.  The  Extend  model  is  a  very  simple  model  and  does  not  include  any 
saturation  effects  such  as  might  occur  if  the  early  adopters  of  Facebook  were  more 
likely  to  find  friends  among  a  given  population  than  were  late  arrivals,  or  if  the 
Facebook  population  began  to  approach  a  limit  of  all  possible  networked  users. 
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Figure  3.  Results  of  the  Extend  Facebook  Model 

Note.  Red  markers  are  model  results,  and  blue  markers  are  Facebook  actual  growth  numbers.  This 
figure  was  created  by  the  authors  using  data  retrieved  from  Facebook. 

The  Eureka  model  begins  with  generation  of  experts  who  initially  are 
assigned  problems  randomly;  the  experts  then  enter  tips  for  solving  each  of  the 
problems.  This  generates  an  initial  set  of  validated  tips.  Other  technicians  are 
generated  next.  The  experts  are  randomly  assigned  new  problems;  they  check  the 
database  for  tips,  and,  if  a  tip  exists,  they  utilize  it  and  solve  the  problem  quickly.  If 
no  tip  exists,  they  take  a  long  time  to  solve  the  problem  and,  with  some  probability, 
either  enter  a  new  tip  or  not.  The  probability  of  entering  a  new  tip  is  given  by  an  S- 
curve  function  that  is  dependent  on  the  number  of  times  a  given  person’s  tips  have 
been  utilized.  This  reflects  the  fact  that  technical  workers  are  highly  motivated  by 
peer  recognition  and  is  consistent  with  Xerox’s  experience. 

The  Eureka  model  was  initially  run  and  the  probability  with  which  technicians 
checked  the  database  was  adjusted  until  a  best  fit  was  obtained  with  real-world  data. 
The  best  fit  occurred  when  the  probability  of  checking  the  database  at  a  given  time 
was  set  to  0.4T/P,  where  T  is  the  number  of  tips  generated  up  to  a  given  time,  and  P 
is  the  total  number  of  problems  that  are  expected  to  be  encountered.  Based  on 
available  data  on  Xerox’s  Eureka  system,  the  initial  number  of  tips  was  100-200 
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(Choo,  n.d.),  the  total  number  of  technicians  during  the  first  five  years  of  use  was 
19,000,  the  number  of  technicians  participating  in  the  Eureka  system  after  five  years 
was  15,000,  and  the  total  number  of  unique  vetted  tips  after  five  years  was  36,000. 
The  total  number  of  problems  to  be  solved  was  not  available;  for  purposes  of 
calibrating  the  model,  the  total  number  of  problems  was  assumed  to  be  50,000.  It 
was  also  assumed  that  it  took  an  average  of  one  hour  to  solve  a  problem  with  a  tip 
and  an  average  of  eight  hours  without  a  tip  and  that  technicians  completed 
approximately  one  trouble  call  per  day.  Jack  Whalen,  a  scientist  who  worked  on 
many  of  Xerox’s  collaborative  information  systems  while  at  Xerox’s  Palo  Alto 
Research  Center  from  1994-2010  (personal  communication,  October  14,  2010) 
estimated  that  non-routine  problems  occurred  more  frequently  than  once  per  week, 
but  less  frequently  than  once  per  day. 

The  most  important  measure  of  effectiveness  of  this  type  of  system  is  the 
participation  rate.  The  participation  rate  drives  the  number  of  new  tips  generated 
over  time  and  is  the  main  factor  in  determining  the  reduction  in  time  to  solve 
problems.  Participation  rates  at  the  end  of  one  year  and  at  the  end  of  five  years,  as 
a  function  of  initial  tips  and  total  expected  problems,  are  shown  in  Figures  4  and  5, 
respectively.  The  results  clearly  show  that  the  ratio  of  initial  tips  to  number  of 
expected  problems  to  be  encountered  is  critical  to  success,  particularly  in  achieving 
a  reasonably  high  rate  of  technician  participation. 
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Figure  4.  Participation  Rate  After  One  Year  as  a  Function  of  Initial 
Tips  and  Number  of  Problems  to  be  Encountered 


Figure  5.  Participation  Rate  After  Five  Years  as  a  Function  of  Initial  Tips 
and  Number  of  Problems  to  be  Encountered 
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IV. 


SoS  Inter-Program  Collaboration  Approach 


There  are  two  parts  to  the  approach  espoused  in  this  work:  developing  a  web- 
based  collaborative  system  and  exploring  contracting  structures  to  facilitate  the 
participation  of  the  individual  system  programs  in  this  system.  Both  make  use  of 
incentives  for  or  attractors  to  the  use  of  the  web-based  system. 

As  discussed  in  the  Introduction  section,  collaboration  among  the  individual 
system  programs  participating  in  an  SoS  acquisition  depends  on  the  presence  of 
mechanisms  to  induce  the  willingness  on  the  part  of  the  individual  programs  to 
collaborate  and  to  enable  their  collaboration.  Mechanisms  can  include  formalized 
structures  for  coordination;  formalized  processes  including  meetings,  deadlines,  etc.; 
sufficient  authority  of  participants;  clarity  of  roles;  and  assets  such  as  personnel  who 
are  dedicated  for  collaboration.  Lateral  mechanisms  can  include  interpersonal 
networks,  effective  communication  and  information  exchange,  technical 
interoperability,  and  training  (Hocevar,  Thomas,  &  Jansen,  2006).  As  discussed  in 
the  Modeling  and  Simulation  of  Collaborative  Systems  section,  a  web-based  system 
is  an  efficient  means  of  providing  a  mechanism  that  provides  many  of  the 
mechanistic  requirements  for  collaboration.  However,  successful  web-based 
collaboration  is  highly  dependent  on  the  value/cost  ratio  that  applies  to  a  given 
system. 

Like  eBay,  Facebook,  and  Eureka,  the  collaborative  system  envisioned  for 
SoS  acquisition  needs  to  have  an  attraction  mechanism — to  attract  the  individual 
programs  to  collaborate  with  the  other  programs  to  achieve  the  objectives  of  the  SoS 
acquisition  program.  Such  a  mechanism,  just  like  those  implemented  with  eBay, 
Facebook,  and  Eureka,  should  be  highly  related  to  the  cost  and  value  of 
collaboration,  as  it  provides  value  to  the  participating  programs  while  at  the  same 
time  a  cost  is  imposed  on  them. 
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Each  individual  program  invariably  is  burdened  with  the  production  of  a 
system  with  required  performance  on  schedule  and  within  budget.  Consequently, 
the  value  and  cost  derived  from  collaborating  with  the  other  programs  are  related  to 
these  parameters — performance,  schedule,  and  budget.  There  is  also,  however, 
another  element  that  can  highly  motivate  participation  in  a  collaborative  system — 
recognition.  Value  is  in  terms  of  recognition.  In  the  Eureka  system,  if  technicians 
see  that  another  worker  has  been  recognized  for  providing  a  tip  for  solving  repair 
problems,  they,  too,  will  want  similar  recognition  and  will  be  motivated  to  enter  a  new 
tip.  If  a  technician  sees  that  his  own  tip  has  been  useful  to  others,  he  will  be 
motivated  to  provide  additional  tips  in  order  to  achieve  further  peer  recognition. 

Thus,  in  addition  to  promoting  value  and  compensating  for  cost,  recognition  should 
be  instituted  for  contributing  to  the  development  of  the  SoS  acquisition.  But,  in  what 
form  should  recognition  be  realized — money,  promotion,  reputation,  rewards  beyond 
a  program  manager’s  tour  on  the  program?  And  to  whom  should  recognition  be 
attributed — just  to  the  program  managers,  or  to  the  entire  team? 

Some  contributors  to  the  cost  of  collaboration,  hence  the  barriers  to 
collaboration,  are  observed.  A  contributor  is  the  cost  of  dedicating  their  resources  to 
developing  the  parts  that  are  required  to  satisfy  the  SoS  requirements.  The  cost  to 
program  personnel  collaborating  in  this  effort  is  the  additional  time  spent  on 
executing  the  SoS  part  of  the  system.  Another  contributor  is  the  cost  associated 
with  a  potential  delay  in  the  development  of  their  own  systems,  caused  by  their 
participation  in  the  SoS  development.  The  individual  programs  that  are  not 
compensated  for  these  costs  will  more  than  likely  decline  to  participate  or  pay  lip 
service  to  collaborating  in  the  SoS  acquisition.  This  problem  can  be  solved  with  new 
contracting  structures,  which  will  be  discussed  in  Section  VI. 

A.  Attraction  Mechanism 

Assessing  value  for  collaborating  is  more  problematic.  There  is  high  value  to 
the  overall  SoS  through  keeping  the  individual  system  programs  aligned  in  order  to 
support  SoS  testing,  but  there  is  not  necessarily  much  value  to  each  individual 
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component  program.  Program  managers  are  typically  rewarded  for  producing  the 
desired  system  on  time  and  within  budget,  but  they  are  not  presently  rewarded  for 
aligning  their  programs  with  other  programs.  Value  to  individual  program  managers 
and  program  offices  must  be  provided  in  order  to  achieve  effective  collaboration. 

The  system  factors  that  determine  the  success  or  failure  of  this  collaborative 
SoS  acquisition  system  include  the  number  of  participating  programs,  which  depend 
on  the  aforementioned  incentives;  the  factors  that  attract  a  collaborator;  the  factors 
that  cause  an  individual  program  to  continue  to  buy  into  collaboration;  the  values  of 
the  SoS  to  the  participating  programs;  and  the  cost  or  risks  to  their  programs. 

The  architecture  of  the  system,  its  development  process,  and  the  organization 
that  manages  the  system  should  “fit”  each  other.  What  an  organization  produces  and 
how  the  organization  is  structured  should  at  least  be  related  to  each  other  (Rechtin, 
2000).  In  order  to  be  effective,  the  structure  of  an  SoS  program  office  must  match 
the  architecture  of  the  SoS  and  the  SoS  development  process.  Currently,  the 
structures  of  most — or  perhaps  all — SoS  program  offices  do  not  match  SoS 
architectures  and  development  processes.  Since  it  is  unrealistic  to  reorganize 
individual  program  offices  in  disparate  Services  into  an  overarching  program  office, 
the  match  can  be  made  by  creating  a  virtual  organization  that  is  linked  by  the  web- 
based  collaborative  system  (WBCS). 

The  AMF(Airborne,  Maritime  and  Fixed)  Joint  Tactical  Radio  contract, 
received  via  personal  communication,  has  been  examined  to  identify  the  Contract 
Data  Requirements  Lists  (CDRLS)  that  are  examples  of  information  required  in  the 
WBCS.  The  AMF  CDRLS  that  impact  the  alignment  of  AMF  with  associated 
systems  include  the  following: 

■  Integrated  master  schedule 

■  Risk/opportunity  management  plan 

■  Interface  design  description 
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Test  plan — AMF  due  not  later  than  (NLT)  90  calendar  days  prior  to 
Critical  Design  Review  (CDR) 


Acceptance  test  plan — due  NLT  240  calendar  days  prior  to  first  GOVT 
EDM  (Engineering  Development  Model)  delivery 

Electromagnetic  environmental  effects  integration  and  analysis 
report — initial  submittal — due  NLT  60  calendar  days  prior  to  CDR 

Electromagnetic  interference  test  report — due  NLT  45  calendar  days 
after  test  event 

Test  procedures — due  NLT  60  days  prior  to  CDR 

Software  requirements  specification 

TEMPEST  control  plan 

TEMPEST  test  plan 

Management  plan 

Configuration  management  plan 

Test  inspection  report — due  NLT  45  calendar  days  after  each  formal 
Contractor  Test  event 


The  required  submission  dates  of  selected  CDRLS  from  the  above  list  are  as 
follows: 

■  Test  plan — due  NLT  90  calendar  days  prior  to  CDR 

■  Acceptance  test  plan — due  NLT  240  calendar  days  prior  to  first 
delivery  of  the  engineering  development  model  to  the  government 

■  Electromagnetic  environmental  effects  integration  and  analysis 
report — initial  submittal  due  NLT  60  calendar  days  prior  to  CDR 

■  Electromagnetic  interference  test  report — due  NLT  45  calendar  days 
after  the  test  event 

■  Test  procedures — due  NLT  60  days  prior  to  CDR 

These  need  times  are  points  where  inputs  are  also  required  from  the  systems 
that  are  a  part  of  the  SoS.  The  general  timeline  for  the  AMF  program  is  shown  in 
Figure  6  with  key  need  points  identified. 
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EM  Interference 
Test  Report1 


EDM 

Delivery 


EM  Jnteg.  & 
Anelysls  Report1 


Test  Plan2 


Test  Procedures3 


Required  Inputs; 

1  Related  programs3  detailed  design  descriptions 

2  SoS  systems  engineering  analysis 

3  Related  programs'  detailed  designs  and  test  plans 


Figure  6.  AMF  Sequence  of  Top-Level  Activities  and  Approximate  CDRL 

Need  Times 

Figure  7  illustrates  the  possible  relative  schedules  of  programs  A  and  B 
whose  systems  A  and  B  are  intended  to  be  part  of  the  AMF  SoS. 


System* 

Program 


Figure  7.  An  Example  of  the  Relative  Sequence  of  Activities  of  the  SoS 

Program  and  Related  Programs 

Just  as  the  integration  of  a  system  requires  the  alignment  of  the  constituent 
components,  a  key  need  for  SoS  alignment  is  to  integrate  the  constituent  systems 
into  the  overall  SoS. 
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Reporting  requirements  to  achieve  effective  alignment  and  SoS  integration 
can  now  be  related  to  attractor  mechanisms  related  to  an  online  collaborative 
system.  Figure  8  shows  a  high-level  view  of  an  online  collaborative  system  and  the 
required  collaborations. 


Systems  Engineering 
Collaboration  (ongoing) 


Collaborative 
^  Scheduling 

t 

Schedule  Inputs 


Collaborative 
^  Test  Planning 

t 

Test  Plan  Inputs 


Figure  8.  High-Level  View  of  an  Online  Collaborative  System  and 

Required  Inputs 

As  aforementioned,  the  attractor  mechanism  that  encourages  participation  in 
an  online  system  depends  on  the  value  to  the  participant  and  the  cost  to  the 
participant. 


The  cost  to  the  SoS  program  office  for  procuring  a  collaborative  software 
system  is  modest  and  such  a  system  is  available  commercially  from  numerous 
vendors,  including  Adobe,  IBM,  Microsoft,  Siemens,  and  Oracle,  among  many 
others.  Typical  products  offer  the  following  features: 


■  Customizable  meeting  rooms 

■  Multiple  meeting  rooms  per  user 

■  VoIP 

■  Audio  integration 

■  Video  conferencing 
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Meeting  recording 


Screen  sharing 


Notes,  chat,  and  white  boarding 


User  management,  administration,  and  reporting 


Polling 


Document  depository  and  content  library 


Hocevar,  Jansen  and  Thomas  (2006)  identifies  the  following  driving  forces  of 
collaboration: 


Formalized  structure  for  coordination 


Formalized  processes,  such  as  meetings 


Interpersonal  networks 


Video  conferencing,  in  particular,  encourages  formation  of  interpersonal 
networks  and  virtual  team  building  through  virtual  face-to-face  meetings. 

The  SoS  program  office  will  also  incur  installation  and  training  costs,  which 
are  typically  modest,  especially  compared  to  typical  SoS  program  budgets.  Costs  to 
the  individual  associated  programs  include  training  costs  and  time  required  to 
participate  in  the  online  system.  There  is  no  additional  documentation  burden  on  the 
individual  systems.  The  collaborative  system  does  not  require  the  individual 
programs  to  input  information  other  than  what  they  are  required  to  generate  under 
their  individual  contracts,  and  the  inputs  do  not  have  to  be  in  any  special  format. 

Value  to  the  SoS  program  office  means  achieving  alignment  and  reducing 
schedule  risk  and  potential  additional  cost.  As  mentioned  in  the  case  of  Xerox’s 
Eureka  system,  systems  engineers  are  likely  to  find  value  through  peer  recognition 
obtained  by  collaborating  with  other  systems  engineers.  Schedulers  and  test 
planners  may  find  the  same  type  of  value  through  peer  interactions.  A  need  for 
technical  workers  is  supported  by  motivational  theory  as  well  as  empirical  evidence, 
such  as  the  Eureka  case.  Maslow’s  motivational  theory  (Maslow,  1943)  supports  the 
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idea  that  technical  workers  who  are  high  on  the  levels  of  satisfied  needs  are 
motivated  by  peer  recognition.  Lawrence  and  Nohria  (2002)  identify  a  four  drives 
theory  of  individual  motivation: 

1.  Drive  to  acquire  (take,  control — objects,  status,  recognition) 

2.  Drive  to  bond 

3.  Drive  to  learn 

4.  Drive  to  defend 

Note  that  the  drive  to  acquire  recognition  is  equivalent  to  Maslow’s  4th-level 
need,  which  is  to  be  held  in  esteem  by  others — a  need  commonly  shared  by 
technical  workers. 

McShane  and  Von  Glinow(2007)  introduce  the  expectancy  theory  of 
motivation: 

■  E-to-P:  An  individual’s  perception  that  his  effort  (E)  will  result  in  a 
particular  level  of  performance  (P). 

■  P-to-O:  An  individual’s  perceived  probability  that  a  certain  level  of 
performance  (P)  will  lead  to  particular  outcomes  (O). 

■  Outcome  valence  is  the  anticipated  satisfaction  that  a  person  feels 
toward  an  outcome;  the  valence  could  be  on  a  scale  of  -1  to  +1  or  -100 
to +100. 

P-to-0  can  be  improved  by  rewarding  high  performance  with  something  that 
the  individual  values  highly.  Since  technical  workers  value  peer  recognition  very 
highly,  participation  by  the  technical  workers  in  a  WBCS  can  be  enhanced  by  the 
simple  expedient  of  providing  high  visibility  in  the  WBCS  for  workers’  effective 
participation,  much  like  in  the  Eureka  system. 

However,  there  is  no  apparent  value  to  the  individual  program  managers.  A 
program  can  be  thought  of  as  an  organization  whose  effectiveness  is  measured  by 
its  ability  to  draw  needed  resources,  mainly  funds  to  keep  the  program  progressing. 
Program  managers  are  rewarded  by  their  ability  to  keep  their  programs  on  schedule 
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and  within  budget;  anything  that  detracts  from  these  goals  is  unwelcome.  Thus,  the 
attractor  mechanism  for  individual  programs  in  an  SoS  development  program  is 
problematic.  There  is  a  cost  to  the  program,  but  no  apparent  value. 

Campbell  (1977)  argues  that  measures  of  organizational  effectiveness 
depend  on  an  organization’s  approach  to  effectiveness.  Campbell  identifies  four 
types  of  approaches.  The  output  goal  approach  emphasizes  end  results,  such  as 
profit  and  quality;  the  internal  process  approach  emphasizes  the  maintenance  of 
effective  human  relationships  within  the  organization;  the  systems  resources 
approach  emphasizes  the  ability  to  draw  needed  resources,  such  as  funding,  from 
its  environment;  and  the  stakeholder  approach  recognizes  the  preferences  of 
various  interest  groups  inside  and  outside  the  organization. 

Program  offices  are  examples  of  organizations  that  fit  the  system  resource 
approach  model.  For  a  program  office  to  succeed,  it  must  obtain  the  necessary 
funding  to  ensure  the  ongoing  development  of  the  system  for  which  it  is  responsible. 
Thus,  program  offices  will  behave  in  a  way  that  maximizes  the  probability  of 
obtaining  continuing  funding  and  perhaps  increased  funding.  Program  offices  will 
strongly  resist  any  behavior  that  jeopardizes  funding. 

Dunlap  (2011)  observes  that  there  was  a  critical  SoS  funding  issue  affecting 
SoS  systems  engineering  in  the  Future  Combat  SoS  program: 

The  system  engineering  strategy  for  requirements  management  at  the  system 
of  systems  level  worked  effectively.  The  requirement  teams  and  the  leads 
were  funded  by  and  under  the  management  of  the  system  of  systems 
engineering  integration  team.  The  system  of  system[s]  engineering 
integration  team  could  require  the  requirements  team  to  work  together  and 
reach  a  consensus.  This  was  not  the  case  for  the  systems’  prime  contractors 
[and]  the  requirements  lead.  The  individual  prime  item  systems  had 
independent  contracts,  as  well  as  varied  management  structures  and  funding 
lines.  The  system  allocation  of  functions  and  requirements  were  never 
successfully  decomposed  or  allocated  from  a  system  of  systems  to 
subsystem[s]/component[s]. 
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The  Future  Combat  System  started  as  a  hierarchical  architecture  for 
decomposing  and  allocating  functions  from  [the]  system  of  systemsto  [the] 
systems  and  then  to  [the]  sub-systems.  Requirements  were  to  be  allocated  at 
each  level  to  maximize  integration  and  reduce  unnecessary  redundancy  and 
systems.  The  program  funding  structure  did  not  allow  for  the  required 
constraints  to  be  mandated  on  the  systems.  The  Future  Combat  System’s 
architecture  resulted  in  [the]  parent  system  [-]of[-]systems  requirements  with 
unacceptable  children  requirements  at  the  system  level. 

A  solution  to  advance  progress  in  resolving  the  issue  on  a  constraint  being 
accepted  at  the  system  and  system  element  levels  of  the  system  of  systems 
would  have  been  readily  available.  Funding  could  have  been  structured  to 
align  with  a  hierarchical  management  organizational  structure. 

Some  program  managers  may  agree  to  participate  in  a  WBCS  without 
additional  funding,  but  to  assure  that  all  program  managers  agree  to  participate, 
funding  should  be  made  available  to  cover  the  additional,  albeit  small,  costs  of 
participating.  Ideally,  additional  funding  should  be  made  to  the  SoS  program  through 
a  contract  modification,  if  necessary,  and  the  SoS  program  office  should  then 
allocate  funds  to  each  of  the  individual  associated  programs  to  cover  their  additional 
costs  of  participating  in  the  web-based  system. 
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V. 


A  Web-Based  Collaborative  System 


One  mechanism  that  holds  promise  for  meeting  many  of  the  requirements  for 
inter-program  collaboration  is  a  web-based  collaborative  system  (WBCS)  on  which 
personnel  from  all  programs  associated  with  an  SoS  can  input  and  retrieve 
information  required  to  align  the  individual  programs. 

A  goal  in  this  research  is  to  explore  the  feasibility  of  such  a  web-based 
system  to  facilitate  the  development  of  an  SoS  through  collaborative  behavior  from 
the  system  programs  that  develop  the  individual  systems  constituting  the  SoS.  The 
purposes  of  this  system  are  to  aid  in  effecting  collaboration  through  management 
and  coordination  of  all  system  programs — the  SoS  program  and  the  system 
programs — and  to  provide  visibility  to  the  progress  of  each  program,  to  problems 
encountered  by  each  program,  and  to  collaborative  decisions  made  by  all  the 
programs  involved.  The  system  is,  thus,  used  to  effect  a  successful  development  of 
the  SoS  as  well  as  all  the  constituent  systems. 

The  web-based  system  proposed  in  this  work  is  based  on  the  concept  of  an 
architecture  for  distributed  and  interoperable  management  of  multi-site  production 
projects  espoused  in  Ishak  et  al.,  (2010).  The  kernel  of  this  system  is  the  SCEP 
(Supervisor,  Customer,  Environment,  Producer)  model  (Archimede  &  Coudert, 

2001).  It  is  based  on  multi-agent  systems  (MAS;  Ferber,  1999).  SCEP  allows  a 
distributed  management  of  acquisition  programs  (i.e.,  system  programs  and  the  SoS 
program)  and  cooperation  via  a  shared  environment  (e.g.,  blackboard)  between  an 
agent  representing  the  SoS  program  and  the  system  program  agents  representing 
the  individual  system  programs  under  the  control  of  a  supervisor  agent.  It  also 
provides  global  visibility  (forecasting  horizon)  of  all  parties  involved,  enabling 
satisfaction  of  the  SoS  program  objectives  and  those  of  the  individual  system 
programs.  In  this  case,  the  customer  is  the  SoS  program,  and  the  producer  is  a 
system  program.  Through  this  web-based  system,  the  management  and 
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coordination  of  the  system  programs  and  the  SoS  program  becomes  that  of  virtual 
enterprises. 

Figure  9  shows  the  architecture  of  the  web-based  system.  As  in  Ishak  et  al. 
(2010),  the  Register  acts  as  an  information  broker.  It  discovers  and  publishes  a 
system  program’s  progress  status  and  the  need  points  (including  progress,  or  lack 
thereof,  issues,  concerns,  milestones,  etc.)  that  affect  the  development  of  the  SoS. 
These  two  functions — discovering  and  publishing — communicate  with  the  Register 
DataBase  (RDB)  containing  the  information  on  the  status  of  all  programs.  Discovery 
here  means  identifying  failures  to  reach  the  need  points  or  to  meet  the  milestones 
required  by  development  of  the  SoS  or  any  of  the  constituent  systems.  Such  need 
points  and  milestones  are  illustrated  in  Figures  6-8. 
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Figure  9.  Web-Based  SoS  Collaborative  System  Architecture 

Note.  This  figure  was  adapted  from  Figure  3  in  Ishak  et  al.  (2010). 


A  System_Program  has  a  system  development  administrator  (SD  Admin), 
which  retrieves  from  the  System  Program  DataBase  (SPDB)  all  information  (e.g., 
needs,  milestones,  and  requirements,  etc.)  pertaining  to  the  SoS  development.  It 
also  stores  in  the  SPDB  the  outcomes  of  the  SP  program’s  activities  in  relation  to  the 
SoS  programs.  A  “publication  agent”  publishes  the  status  of  the  system  program 
(i.e.,  need  points  or  milestones)  through  its  interaction  with  the  Register. 

The  SoS_Program  has  the  following  components  —  the  SoS  Program 
Manager,  the  SoS  Program  DataBase  (SoSPDB),  the  SoS  Program  Management 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-29- 


System,  an  SoS_Discovery  module,  and  the  Local  Register.  The  SoS  Program 
Manager  is  responsible  for  the  SoS  program  realization.  The  SoSPDB  contains  the 
description  of  the  SoS  program,  the  status  of  the  system  programs,  a  local  register, 
and  the  program  management  system.  The  SoS_Discovery  module  obtains  from 
the  Register  the  status  of  all  system  programs  and  stores  it  at  the  Local  Register.  A 
limited  copy  of  the  Register,  the  Local  Register  stores  information  about  the 
system’s  registered  concerns  and  progress.  It  accelerates  the  discovery  of  issues 
and  aids  the  SoS  program  in  managing  the  system  programs. 

To  enhance  the  collaboration  between  the  SoS  program  and  the  system 
programs,  the  information  in  the  Local  Register  and  the  Register  must  be 
synchronized  and  verified  for  aging  and  accuracy.  The  SoS  Program  Management 
System  allows  for  distributed  management  of  the  collaboration  of  the  SoS  program 
with  the  system  programs.  The  SCEP  supervisor  agent  obtains  details  of  the  SoS 
program  from  the  SPDB.  It  also  creates  the  shared  environment,  the  SoS  program 
agent  representing  the  SoS  program,  and  the  SCEP  ambassador  agents 
representing  the  system  programs.  As  elaborated  in  Ishak  et  al.  (2010),  the  agents 
of  the  SCEP  model  are  integrated  into  the  SoS  Program  Management  System  so  as 
to  manage  the  execution  of  the  system  programs  in  a  distributed  and  autonomous 
manner.  Through  the  shared  SCEP  environment,  the  management  of  the  programs 
is  effected  by  cooperation  between  the  SoS  program  agent  and  the  ambassador 
agents  in  resolving  any  issues  and  concerns  from  the  system  programs. 

The  first  step  of  the  management  process  involves  the  identification  of  a 
system  program  that  has  concerns  and  whose  status  has  been  published  and 
discovered.  The  second  step  deals  with  instantiation  of  SCEP  components  on  the 
SoS_Program  side  and  connection  to  the  System_Program  identifed  in  the  first  step. 
The  SCEP  supervisor  agent  creates  the  shared  environment  as  well  as  the  SoS 
program  and  ambassador  agents.  The  last  step  concerns  interactions  and 
cooperation  between  the  SoS_Program  and  its  System_Programs  through  the 
SCEP  instantiation. 
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At  the  beginning  of  the  SoS  program,  through  the  supervisor  agent,  the  SoS 
program  agent  deposits  the  SoS  program  requirements  in  the  SCEP  environment. 
Invited  by  the  supervisor  agent,  every  ambassador  agent  pulls  from  the  environment 
the  SoS  requirements  and  constraints  on  the  individual  system  program  it 
represents.  After  the  collection  of  information,  the  ambassador  agent  generates  the 
requirements  for  the  corresponding  system.  The  ambassador  agent  receives  the 
status  and  concerns  from  the  corresponding  system  program  and  deposits  them  in 
the  SCEP  environment  to  be  discovered  by  the  SoS  program  agent  and  other 
ambassador  agents.  The  lack  of  progress  of  each  system  program  will,  thus,  be 
visible  to  all  parties,  and  timely  corrective  actions  can  then  be  taken. 

Furthermore,  as  this  architecture  might  underlie  commercially  available  tools, 
this  work  does  not  suggest  that  a  new  software  tool  be  developed  to  implement  this 
architecture.  This  architecture  might  be  used  as  a  guide  to  aid  in  identifying  any  of 
the  commercially  available  tools  for  use  in  effecting  collaboration  among  the 
programs  in  an  SoS  development  effort. 
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VI. 


Enabling  Contracting  Structures 


As  discussed  earlier  in  this  report,  a  lack  of  alignment  and  a  lack  of 
collaboration  between  and  among  the  individual  system  programs  are  two  important 
issues  leading  to  problems  in  SoS  acquisition,  and  a  web-based  SoS  collaborative 
system  can  be  developed  to  support  DoD  SoS  acquisition  programs.  This  section 
now  presents  how  contracting  structures,  more  specifically,  contract  management 
processes,  can  facilitate  the  use  of  the  web-based  SoS  collaborative  system  by  the 
SoS  individual  system  programs.  The  context  of  the  contract  management  process 
is  used  to  illustrate  these  structures. 

Typically,  the  contract  management  process  is  discussed  from  two 
perspectives — the  pre-contract  award  phase  and  the  post-contract  award  phase. 
However,  to  provide  additional  granularity  and  a  deeper  level  of  analysis,  it  is 
appropriate  to  discuss  the  process  using  a  six-phase  life  cycle.  These  six  phases  of 
contract  management  for  the  procuring  organization  consist  of  Procurement 
Planning,  Solicitation  Planning,  Solicitation,  Source  Selection,  Contract 
Administration,  and  Contract  Closeout  (Rendon  &  Snider,  2008).  Each  of  these 
contract  management  life  cycle  phases  involves  specific  contracting  activities  that 
can  facilitate  the  use  of  a  web-based  collaboration  system.  Given  the  SoS  context 
of  this  research,  these  contract  management  activities  would  be  performed  by  any 
one  of  the  individual  system  program  offices. 

Procurement  Planning  involves  the  process  of  identifying  which  business 
needs  can  best  be  met  by  procuring  products  or  services  outside  the  organization. 
This  process  involves  determining  whether  to  procure,  how  to  procure,  what  to 
procure,  how  much  to  procure,  and  when  to  procure  (Rendon  &  Snider,  2008).  This 
phase  of  the  contracting  process  includes 

•  conducting  outsource  analysis; 
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•  determining  and  defining  the  requirement  (the  supply  or  service  to 
procure); 

•  conducting  market  research  and/or  a  pre-solicitation  conference; 

•  developing  preliminary  requirements  documents  such  as  work 
breakdown  structures  (WBS),  statements  of  work  (SOW),  performance 
work  statement  (PWS),  or  other  descriptions  of  the  supply  or  service  to 
be  procured; 

•  developing  preliminary  budgets  and  cost  estimates; 

•  preliminarily  considering  contract  type  and  any  special  contract  terms 
and  conditions;  and 

•  conducting  a  risk  analysis. 

The  procurement  planning  activities  for  each  individual  system  program  must 
be  collaborated  on  and  aligned  with  the  acquisition  objectives  of  the  SoS  acquisition 
program.  Specific  activities  such  as  defining  the  requirement,  developing  contract 
statements  of  work,  determining  system  specifications,  and  conducting  a  risk 
analysis  should  be  performed  in  collaboration  and  alignment  with  the  other 
acquisition  programs  within  the  SoS  program.  The  use  of  a  web-based  collaboration 
system  accessible  by  the  SoS  program  and  individual  system  programs  would 
facilitate  this  collaboration  and  alignment. 

Solicitation  Planning  involves  the  process  of  preparing  the  documents  needed 
to  support  the  solicitation.  This  process  involves  documenting  program 
requirements  and  identifying  potential  sources  (Rendon  &  Snider,  2008).  This 
process  includes 


•  determining  procurement  method  (sealed  bids,  negotiated  proposals, 
e-procurement  methods,  procurement  cards,  etc.); 

•  determining  contract  type  (fixed-price  versus  cost  type); 

•  developing  the  solicitation  document  (IFB,  RFQ,  or  RFP); 

•  determining  proposal  evaluation  criteria  and  contract-award  strategy; 

•  structuring  contract  terms  and  conditions;  and 
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•  finalizing  the  solicitation,  WBS,  SOW,  and  product  or  service 
descriptions. 

The  above  solicitation  planning  activities  reflect  the  results  of  the  procurement 
planning  process.  As  the  system-of-systems  program  office  begins  developing  the 
solicitation  documents  (for  example,  the  WBS,  SOW,  specifications,  etc.), 
collaboration  and  alignment  with  the  other  acquisition  program  offices  making  up  the 
system  of  systems  take  on  an  increased  importance.  The  use  of  a  web-based 
collaboration  system  accessible  by  the  SoS  and  individual  system  programs  would 
facilitate  this  much-needed  collaboration  and  alignment. 

Once  the  solicitation  (for  example,  the  Request  for  Proposal)  is  completed, 
the  Solicitation  phase  is  the  process  of  issuing  or  deploying  the  solicitation  and 
obtaining  information  bids  or  proposals  from  the  offerors  on  how  project  needs  can 
be  met  (Rendon  &  Snider,  2008).  This  process  includes 

•  conducting  advertising  of  the  procurement  opportunity,  or  providing 
notice  to  interested  offerors; 

•  conducting  a  pre-proposal  conference,  if  required;  and 

•  developing  and  maintaining  a  qualified  bidders  list. 

For  the  Solicitation  process,  many  government  agencies  have  established 
web-based  systems  for  centralizing  and  providing  the  maximum  visibility  for  the 
advertisement  of  procurement  opportunities  to  industry.  This  ensures  a  level  of 
integrity,  accountability,  and  transparency  in  the  contracting  process.  For  the  United 
States,  federal  government  contracting  opportunities  are  publicized  through  the 
Government  Point  of  Entry  (GPE),  which  is  the  single  point  where  government 
business  opportunities  greater  than  $25,000,  including  synopses  of  proposed 
contract  actions,  solicitations,  and  associated  information,  can  be  accessed 
electronically  by  interested  offerors  (www.fedbizopps.gov). 
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Source  Selection  is  the  process  of  receiving  bids  or  proposals  and  applying 
the  proposal  evaluation  criteria  to  select  a  supplier  (Rendon  &  Snider,  2008).  The 
source  selection  process  includes  the  evaluation  of  offers  and  proposals,  and 
contract  negotiations  between  the  buyer  and  the  seller  in  attempting  to  come  to 
agreement  on  all  aspects  of  the  contract,  including  cost,  schedule,  performance, 
terms  and  conditions,  and  anything  else  related  to  the  contracted  effort.  This 
process  includes 

•  applying  evaluation  criteria  to  management,  cost,  and  technical  bids  or 
proposals; 

•  negotiating  with  suppliers;  and 

•  executing  the  contract  award  strategy. 

The  complexity  of  the  source  selection  process  will  depend  on  the  contract 
award  strategy  selected.  The  complexity  and  challenges  of  the  contract  source 
selection  process  will  depend  on  whether  the  government  uses  a  price-directed 
award  strategy  (for  example,  lowest  price/technically  acceptable)  or  a  trade-off 
process  (for  example,  technical  approach  is  more  important  than  price)..  In  some 
contract  source  selections,  in  which  the  requirement  is  clearly  definable  and  the  risk 
of  unsuccessful  contract  performance  is  minimal,  cost  or  price  may  play  a  dominant 
role.  In  other  source  selections,  in  which  the  requirement  is  less  definitive  and  more 
development  work  is  required  (resulting  in  greater  performance  risk),  more  technical 
or  past  performance  considerations  may  play  a  dominant  role. 

If  the  individual  system  program  requirement  (supply  or  service  being 
procured)  will  have  a  significant  impact  on  the  SoS  acquisition  program,  it  will  be 
essential  for  the  other  programs  to  be  involved  in  the  source  selection  process, 
especially  the  evaluation  of  offeror  cost,  schedule,  and  technical  proposals,  as  well 
as  past  performance  evaluation.  The  use  of  a  web-enabled  collaboration  system  will 
facilitate  the  required  integrated  assessment  of  offeror  capabilities  identified  during 
this  process. 
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Once  the  contract  is  awarded,  the  contract  administration  phase  begins. 
Contract  administration  is  the  process  of  ensuring  that  each  party’s  performance 
meets  the  contractual  requirements  (Rendon  &  Snider,  2008).  The  activities 
involved  in  contract  administration  will  depend  on  the  contract  statement  of  work, 
contract  type,  and  contract  performance  period.  The  contract  administration  process 
typically  includes 


•  conducting  a  pre-performance  conference, 

•  monitoring  the  contractor’s  work  results, 

•  measuring  the  contractor’s  performance,  and 

•  managing  the  contract  change-control  process. 


In  major  defense  acquisition  projects,  such  as  SoS  acquisition  programs,  the 
contract  administration  phase  is  critical  to  effective  project  management.  In  this 
phase  of  the  contract  management  process,  the  contractor  is  performing  the 
statement  of  work  requirements,  and  the  completed  work  is  then  measured  and 
evaluated  by  the  buying  organization. 

A  significant  aspect  of  the  contract  administration  phase  consists  of 
monitoring  the  contractor’s  performance.  The  monitoring  and  controlling  project 
management  processes  are  focused  on  ensuring  that  project  objectives  are  met  as 
the  project  manager  performs  such  activities  as  measuring  progress  against  plan, 
holding  status  meetings,  and  correcting  the  divergences  from  schedule  or  budget. 
Another  major  emphasis  of  contract  administration  activities  is  measuring  contractor 
performance.  The  contractor’s  performance  is  measured  to  ensure  that  the  actual 
contractor  work  results  meet  the  cost,  schedule,  and  performance  standards  agreed 
to  in  the  contract.  In  addition,  it  would  be  naive  to  think  that  once  the  contract  is 
awarded,  the  contract  will  never  need  to  be  changed  or  modified  during  the  contract 
period.  Contracts  frequently  require  changes  due  to  various  reasons  during  the 
period  of  performance.  Another  major  part  of  contract  administration  activities  is 
focused  on  managing  the  contract-changes  process. 
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Once  again,  in  the  SoS  context,  given  the  high  risk  and  complexities  of  SoS 
acquisition  programs,  it  will  be  essential  for  the  other  program  offices  to  be  involved 
in  the  contract  administration  process,  especially  the  monitoring,  controlling,  and 
measuring  of  the  contractor’s  performance,  as  well  as  the  coordination,  review,  and 
approval  of  contract  changes.  The  use  of  a  web-enabled  collaboration  system  will 
facilitate  the  required  integrated  administration  of  the  SoS  contracts. 

Typically,  an  acquisition  contract  can  end  in  one  of  three  ways:  First,  the 
contract  can  be  successfully  completed  and  allowed  to  run  its  full  period  of 
performance,  and  then  closed  out;  second,  the  contract  can  be  terminated  for  the 
convenience  of  the  government;  or  third,  the  contract  can  be  terminated  for  default. 
Regardless  of  how  the  contract  ends,  in  the  end,  all  contracts  must  be  closed  out. 
Thus,  the  final  phase  of  the  contracting  process  is  the  Contract 
Closeout/Termination  phase.  Contract  Closeout  is  the  process  of  verifying  that  all 
administrative  matters  are  concluded  on  a  contract  that  is  otherwise  physically 
complete  (Rendon  &  Snider,  2008).  The  closeout  of  contracts  that  are  physically 
complete  requires  the  verification  of  documentation  that  reflects  the  completion  of  all 
required  contractual  actions.  A  contract  is  considered  to  be  physically  completed 
when  the  contractor  has  completed  the  required  deliveries  and  the  government  has 
inspected  and  accepted  the  supplies;  the  contractor  has  performed  all  services,  and 
the  government  has  accepted  the  services;  and  all  contract  option  provisions  have 
expired. 

This  contract  closeout  process  includes  the  following  activities: 

•  final  inspection  and  acceptance  of  products  or  services, 

•  processing  of  government  property  dispositions, 

•  final  contractor  payments,  and 

•  documentation  of  the  contractor’s  final  past-performance  report. 
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The  contract  closeout  process  for  each  individual  system  program  must  be 
collaborated  on  and  aligned  with  the  acquisition  objectives  of  the  SoS  acquisition 
program.  In  the  SoS  context,  specific  activities,  such  as  final  inspection  and 
acceptance  of  products  or  services,  should  be  performed  in  collaboration  and 
alignment  with  the  other  acquisition  programs  within  the  SoS.  Once  again,  the  use 
of  a  web-based  collaboration  system  accessible  by  the  SoS  and  individual  system 
program  offices  would  facilitate  this  collaboration  and  alignment. 

The  contract  management  process  is  a  critical  aspect  of  a  system  acquisition 
program.  It  is  typically  the  effectiveness  of  the  contracting  process  that  determines 
the  success  of  an  acquisition  program.  SoS  acquisition  programs  entail  a  higher 
level  of  complexity  and  risk,  which  necessitates  the  need  for  collaboration  and 
alignment  among  the  individual  system  programs.  The  use  of  the  web-based  SoS 
collaborative  system  by  the  SoS  individual  system  programs  can  facilitate  this 
collaboration  and  alignment. 
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VII. 


Conclusion 


System-of-systems  (SoS)  acquisition  research  has  identified  lack  of  alignment 
and  lack  of  collaboration  as  two  important  issues  leading  to  problems  in  SoS 
acquisition.  This  report  captures  the  exploratory  work  toward  improving  alignment 
between  and  collaboration  among  the  individual  system  programs  in  the 
development  of  an  SoS. 

An  SoS  inter-program  collaboration  approach  is  proposed.  It  is  inspired  by 
some  existing  web-based  collaborative  systems,  such  as  eBay,  Facebook,  and 
Eureka,  and  suggests  an  attraction  mechanism  to  effect  SoS  inter-program 
collaboration.  In  addition,  a  web-based  collaborative  system  is  also  suggested. 

Based  on  an  architecture  for  distributed  and  interoperable  management  of  multi-site 
production  projects,  it  allows  personnel  of  all  programs  associated  with  an  SoS  to 
input  need  points  for  component  system  inputs  and  retrieve  information  required  to 
align  the  individual  programs.  This  architecture  might  be  used  as  a  guide  to  aid  in 
identifying  any  of  the  commercially  available  tools  for  use  in  effecting  collaboration 
among  the  programs  in  an  SoS  development  effort. 

Furthermore,  as  part  of  this  SoS  inter-program  collaboration  approach, 
contracting  structures  for  facilitating  collaboration  among  the  system  programs  are 
also  considered. 

Finally,  this  work  forms  a  basis  for  implementing  a  web-based  SoS 
collaborative  system  to  support  DoD  SoS  acquisition  programs. 
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